1. Field of the Invention
The present invention generally relates to special purpose data processing circuits and, more particularly, to data processing circuits for image data in automated inspection systems.
2. Description of the Prior Art
Image processing for display generation or feature measurement, extraction or recognition is perhaps the most computation and storage intensive category of data processing problem commonly encountered at the present time. Feature measurement, such as in measurement of cell features in biological applications and the monitoring of coating or layering processes are two examples of applications of automatic feature measurement which require high-speed computation of high accuracy feature measurement data. While the actual processing may be quite simple in some cases, such problems typically involve relatively massive amounts of data. For example, consider that the smallest unit of an image for processing is referred to as a pixel with which the image surface is tiled, often in the form of a matrix. If the resolution of the image is a matrix of one thousand pixels on a side, the complete image will include one million pixels. Each pixel may further contain data representing image values such as color, hue and saturation and other data which represents the relationship to other pixels such as an object number or a location in a three-dimensional scene. Transparency and translucency are often considered and numerous objects in the line of sight may each contribute to the image pixels value and data representing these objects carried with the pixel information. The number of bits of such image value information must be multiplied by the number of pixels in the image to obtain the number of bits necessary to represent the image. Since some operation may be required on any or all of this image data, it can be understood that many millions of data processing operations are required for even modest degrees of spatial and image value resolution.
Accordingly, many approaches have been tried in recent years to reduce the amount of data which must be processed in order to achieve desired levels of throughput from image processing apparatus. Nevertheless, even with high-speed computers, possibly including special purpose co-processors and pipelined architectures, it is not uncommon for the processing of a single image to require several hours to complete. In the field of automated inspection systems, such processing times present a major limitation on throughput of manufacturing processes. However, in some manufacturing fields, such as the manufacturing of electronic circuit devices at high integration densities, there is no viable alternative to automated inspection.
Therefore, the art of image feature measurement, extraction and/or recognition has generally advanced through the development of special purpose image transducers and processing arrangements which are specifically adapted to particular kinds of image features. For example, U.S. Pat. No. 4,424,588 to Satoh, describes processing to detect the position of a symmetrical article. U.S. Pat. No. 4,499,597, to Alves, describes centroid accumulation for small object detection. Pixel values are compared among adjacent pixels to determine the maximum image value pixel in a character segment in U.S. Pat. No. 4,625,330, to Higgins and a similar neighborhood comparison is disclosed in Gennery, U.S. Pat. No. 4,703,513; both being directed to the enhancement of video signals. An arrangement for imaging a three-dimensional device applied to lead frame assembly is disclosed in U.S. Pat. No. 5,030,008, to Scott et al. An optical system for distance measuring is disclosed in U.S. Pat. No. 5,054,926, to Dabbs et al. Some exemplary data processing arrangements for use in image processing systems are also disclosed in U.S. Pat. Nos. 5,016,173, to Kenet et al,, 4,918,636, to Iwate et al., 4,963,018, to West, 4,979,221, to Perryman et al., 4,925,302, to Cutler, 4,845,356, to Baker, 4,818,110, to Davidson, and 4,707,610, to Ludlow et al.
This latter patent to Ludlow et al. is directed to the measuring of surface profiles and line width measurements in regard to the manufacture of integrated circuit devices. The wafer to be measured is mounted for oscillatory movement and an optical system focusses a beam on a small spot on the surface. The spot is scanned along the wafer while the focus is progressively changed to derive a series of samples of the surface profile.
More recently, however, it has become desirable to measure a plurality of reflecting profiles within a body of material such as would be presented by a plurality of layers of a semiconductor structure. In such imaging, it may not be possible to, say, follow each profile separately and differentiation of profiles may be difficult. Also, since multiple scanning is relatively slow and potentially could engender positional errors in connection with the different surfaces imaged, it is desirable to sense all profiles in a single scan of the object to be imaged.
As a solution to the above imaging problems, a confocal imaging system is disclosed in U.S. patent application Ser. No. 07/871,458, filed Apr. 21, 1992, now U.S. Pat. No. 5,248,816 entitled TANDEM LINEAR SCANNING CONFOCAL IMAGING SYSTEM, by Wu et al, assigned to the assignee of the present invention and hereby fully incorporated by reference. The system disclosed therein is capable of imaging a plurality of surfaces or partially reflective boundaries within a volume of material simultaneously by utilization of confocal imaging principle which will be discussed in greater detail below.
At the present state of the art in optical transducers (e.g. electronic cameras) as disclosed therein, spatial resolution over the surface (e.g. the x and y directions) of the device and within (e.g. in the z direction) the device is on the order of a few microns or less. Therefore, even a small surface such as that of a chip may involve several million pixels and the profile information may desirably reach the resolving of, say, eight surfaces at eight bits dimensional accuracy. Bit streams of data from each surface are essentially derived simultaneously, in parallel, for each pixel. Accordingly, it is seen that an extremely great quantity of data can be captured. This quantity of data must therefore be stored and processed rapidly to exploit the capabilities of transducers presently available and to accomplish the desired imaging.
Consider also that some attempts to increase processing speed have involved truncation of data. However, such truncation effectively discards information which is present in the original signals from a transducer device. Even truncation is of extremely limited value in reducing data since it is now common to produce integrated circuits having several millions of components, each of which must be imaged with sufficient resolution for meaningful inspection, requiring several pixels for each component or feature of a component. Therefore, data truncation, even to one bit per pixel, is very much limited by the minimum amount of data which is required to achieve the desired inspection function. Further, since data is discarded in the truncation process, the resolution capabilities of transducers would not be fully exploited if data truncation were to be performed.
In summary, the present state of the data processing art does not allow real time data acquisition from an optical transducer which fully exploits the capabilities of transducers which are now possible or allow real time processing of such large amounts of data as are required for a desired degree of optical resolution and with sufficient throughput to provide an automated, real time, inspection system suitable for present manufacturing systems for high density integration electronic components.
A partial solution to the computational problems discussed above is disclosed in U.S. patent application Ser. No. 07/999,323 (Attorney's Docket Number FI9-91-227) now U.S. Pat. No. 5,455,899, filed concurrently herewith, entitled HIGH SPEED IMAGE DATA PROCESSING CIRCUITRY by Donald C. Forslund, which is also assigned to the assignee of the present invention and fully incorporated by reference herein. This high speed image data processing circuitry uses parabola fitting to obtain high resolution height values of a plurality of partially reflecting surfaces within a volume and to output these height values in parallel and in real time in synchronism with the scanning of a camera across the sample being inspected.
The camera used for confocal imaging preferably uses a charge coupled device (CCD) for image capture since such devices are highly responsive to small changes in illumination (e.g. pixel brightness values), on which confocal imaging relies and are easily calibrated and output signal values corrected for so-called fixed pattern noise which occurs because of manufacturing irregularities in CCD devices themselves. CCD's are also attractive since they are essentially self-scanning in response to a multi-phase clock applied thereto and complex scanning circuitry is not required. The self-scanning features of CCD's also can improve the resolution of the devices since very little, if any, image space is consumed by electrical connections to the photosites or switching devices located within the image array, although this is of little significance in the preferred embodiment since a sparse array of photosites is used in the CCD of the preferred embodiment of the camera; that is, the photosites are not contiguously positioned in the preferred CCD.
Nevertheless, the self-scanning nature of CCD's prevents the possibility of massively parallel outputs therefrom. In fact, while a CCD may, in theory be articulated into as many segments as desired and an output taken simultaneously from each segment, the physical layout of CCD's usually prevents more than four segments and effectively prohibits interior segments which do not reach the edge of the imaging area of the device. Therefore, CCD's are not well-suited to providing a plurality of outputs in parallel.
This problem is further complicated by the confocal imaging process, itself, since, as will be explained in greater detail below, when one surface at a given height is being imaged by a particular photosite of the CCD, an adjacent photosite of the CCD will be responsive to a different height and at a different position on the imaged sample. Therefore, even if outputs could be brought out in parallel from every photosite of the CCD, this information would be in a format far different from that required by the high speed data processor disclosed in the above-incorporated application since information regarding surface heights at a particular location require simultaneous presentation of all data from a single sample location. Specifically, in order to perform parabola fitting to obtain high resolution height data, a plurality of intensity values imaged at different heights from the same location on the sample must be present at the same time (e.g. in parallel). Data must be reorganized in location and time-shifted in order to derive useful image information. Time shifting itself is known in the art but usually involves latching or sample-and-hold arrangements which would be prohibitive for the number of photosites from which data is collected from the CCD. Further, since the output from a plurality of photosites of the CCD are necessarily serialized, some commutation between such latches or sample-and-hold circuits would be necessary. Reorganizing in location could also be done by a commutation arrangement but either a large number of commutators would be required to operate in parallel and in synchronism with the time shifting arrangement or a further sampling or latching arrangement would be required. The provision of two cascaded latching stages would, of course, effectively prohibit operation in real time and, in any event, would be very hardware intensive. Therefore it is seen that interfacing from any CCD used in a confocal imaging environment is extremely complex, particularly when it is desired to capture large quantities of data in substantially real time.